Improved Phone Posterior Estimation through K-nn and Mlp-based Similarity

I would like to thank Professor Hervé Bourlard, my master's thesis supervisor, who allowed me to do it at the Idiap Research Institute and who gave me a very interesting work subject: Improved Phone Posterior Estimation Through k-NN And MLP-based Similarity. I thank him, and also Dr Mathew Magimai Doss and Mrs Afsaneh Asaei for their kindness, their guidance, their availability and their support all along this thesis. I am also thankful to Professor Thierry Dutoit, my supervisor at Faculté Polytechnique de Mons in Belgium, for giving me the opportunity to realize my master's thesis within the international scientific and cultural context of the Idiap Research Institute. I would like to thank also particularly Mrs Nadine Rousseau and Mrs Sylvie Millius for their kindness, their advices and their helps in all the administrative procedures and in the research of an apartment in Martigny. Finally, I would like to thank all the people working in Idiap Research Institute for their warm welcome, when I came there for the first time and throughout my master's thesis, and This work was performed in the context of a European ERASMUS exchange program, complemented by the AMIDA training program. AMIDA (Augmented Multi-party Interaction with Distance Access) is a European Union Integrated Project, contract number IST-033812. The authors gratefully thank the EU for their financial support, and all project partners for a fruitful collaboration. More information about AMIDA is available from the project web site www.amiproject.org.Abstract In this work, we investigate the possible use of k-nearest neighbour (kNN) classifiers to perform frame-based acoustic phonetic classification, hence replacing Gaussian Mixture Models (GMM) or MultiLayer Perceptrons (MLP) used in standard Hidden Markov Models (HMMs). The driving motivation behind this idea is the fact that kNN is known to be an "optimal" classifier if a very large amount of training data is available (replacing the training of functional parameters by plain memorization of the training examples) and the correct distance metric is found. Nowadays, amount of training data is no longer an issue. In the current work, we thus specifically focused on the "correct" distance metric, mainly using an MLP to compute the probability that two input feature vectors are part of the same phonetic class or not. This MLP output can thus be used as a distance metric for kNN. While providing a "universal" distance metric, this work also enabled us to consider the speech recognition problem under …